Device pairing with optical codes

ABSTRACT

Embodiments for device pairing using optical codes are described. One embodiment is a wearable device with an image sensor configured to capture an image including a first optical code from a first host device. The wearable device decodes the first optical code, and in response to the first optical code, initiates broadcast of a pairing advertisement. The host device displays a second optical code in response to the pairing advertisement, and the wearable device captures and processes the second optical code to determine a host pairing advertisement code. The wearable device then, in response to the second optical code, initiate broadcast of a second pairing advertisement including the host pairing advertisement code. In various embodiments, a secure wireless channel is then established and used for further secure communications.

CLAIM OF PRIORITY

This application is a continuation of U.S. Pat. Application Serial No.16/947,339, filed on Jul. 29, 2020, which is a continuation of U.S. Pat.Application Serial No. 15/716,124, filed on Sep. 26, 2017, which claimsthe benefit of priority to U.S. Provisional Application Serial No.62/399,901, filed on Sep. 26, 2016, each of which is hereby incorporatedby reference herein in their entireties.

TECHNICAL FIELD

Embodiments of the present disclosure relate generally to devicecommunications, and particularly to the use of optical communicationsfor pairing of a wearable device (e.g. smart glasses) with a host device(e.g. a smart phone).

BACKGROUND

Eyewear, especially those carrying onboard electronics, can be bulkywith significant size and weight tradeoffs associated withcommunication, battery, and charging systems, making them difficult towear and charge. Additionally, given limited onboard resources,including limited interface elements, establishing secure communicationlinks can be difficult.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

To easily identify the discussion of any particular element or act, themost significant digit or digits in a reference number refer to thefigure(“FIG.”) number in which that element or act is first introduced.

FIG. 1A is a block diagram showing an example messaging system forexchanging data (e.g., messages and associated content) over a network,in accordance with some embodiments.

FIG. 1B is a block diagram showing an example messaging system forexchanging data (e.g., messages and associated content) over a networkwith a wearable device (e.g. companion) via a client device (e.g. host),in accordance with some embodiments.

FIG. 2 illustrates a system for recognizing optical codes, in accordancewith some embodiments.

FIGS. 3A and 3B illustrate aspects of optical codes, in accordance withsome embodiments.

FIGS. 4A and 4B illustrate capture of optical codes using a wearabledevice, in accordance with some embodiments.

FIG. 5 illustrates a system for secure device pairing using opticalcodes, in accordance with some embodiments.

FIGS. 6 and 7 illustrate aspects of optical code recognition, inaccordance with some embodiments.

FIGS. 8 and 9 illustrate aspects of optical code recognition, inaccordance with some embodiments.

FIG. 10 illustrates a process flow for device pairing with opticalcodes, in accordance with some embodiments.

FIG. 11 illustrates a process flow for device pairing with opticalcodes, in accordance with some embodiments.

FIGS. 12A-12B show aspects of a host device that may use optical codepairing, in accordance with some embodiments described herein.

FIG. 13 shows aspects of a wearable device that may use optical codepairing, in accordance with some embodiments described herein.

FIG. 14 is a block diagram illustrating a representative softwarearchitecture, which may be used in conjunction with various hardwarearchitectures herein described.

FIG. 15 is a block diagram illustrating components of a machine,according to some example embodiments, able to read instructions from amachine-readable medium (e.g., a machine-readable storage medium) andperform any one or more of the methodologies discussed herein.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques,instruction sequences, and computing machine program products asillustrative embodiments of the disclosure. In the followingdescription, for the purposes of explanation, numerous specific detailsare set forth in order to provide an understanding of variousembodiments of the inventive subject matter. It will be evident,however, to those skilled in the art, that embodiments of the inventivesubject matter may be practiced without these specific details. Ingeneral, well-known instruction instances, protocols, structures, andtechniques are not necessarily shown in detail.

Quick Response (QR) codes, and other optical barcodes, are a convenientway to share small pieces of information to users of mobile devices,wearable devices, and other smart devices. Embodiments described hereinuse optical codes for pairing a wearable device with a host device.Wearable devices generally have limited battery and processingresources. For wearable devices that include a camera, the function ofthe camera and quick camera response times are often a key userpreference, while wireless data transfer and communications may besecondary. In such an environment, the use of optical codes to initiatesecure device pairing provides benefits related to the use of the cameraas prioritized over the use of wireless communications. Additionally,since optical code processing may be part of a wearable device, the useof these processing features may not involve a large use of additionalwearable device resources.

Embodiments described herein may operate where a host device (e.g.,smart phone) displays an optical code, and a wearable device captures animage of the optical code. The wearable device processes the code, andthen in response, broadcasts a wireless pairing signal. Any local hostdevice can receive the pairing signal and generate a second optical codebased on the pairing signal. The wearable device then captures thesecond optical code and uses information from the second optical code toestablish a secure wireless connection, which is used for securehandshaking.

Further, some embodiments may use specialized optical codes. Typically,an optical barcode uses a finder pattern for identification of theoptical barcode. Conventional finder patterns commonly use multiplegeneric markings conspicuously placed within the optical barcode. Suchconspicuous and generic markings can be unsightly and often serve nopurpose other than to function as a finder pattern.

QR codes, and other optical barcodes (e.g., Universal Product Code (UPC)barcodes, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, UltraCode), are a convenient way to share small pieces of information withusers of mobile devices, wearable devices, and other smart devices. Forinstance, QR codes are two-dimensional optical barcodes that encodeinformation readable by a device (e.g., a smart phone) equipped with acamera sensor. Typically, a QR code includes one or more functionalpatterns, such as a finder pattern, used for identification andrecognition of the QR code or an alignment pattern used to facilitatedecoding. Conventional finder patterns comprise multiple markings thatare generic in design such as square marks placed in all corners exceptthe bottom right corner (as is the case with a QR code). These finderpatterns are absent aesthetic elements such as curves, nonuniformities,and other stylistic elements and often conform to a particular standardto promote open use of the optical barcode.

Embodiments described herein relate to the use of such codes for securepairing of two electronic devices. For example, a phone may display anoptical code, and a camera on a peripheral device (e.g. glasses or otherwearable device) captures the optical code, and then uses the opticalcode as part of a set of wireless pairing communications to establish asecure wireless link between the two devices. In various exampleembodiments, an optical barcode that uses custom or non-standardfunctional patterns provides users with an aesthetically pleasing,branded barcode that allows for an exclusive experience associated withthe optical barcode. For example, an entity logo (e.g., a logo of acompany, organization, or individual) can be used as a finder patternand, in some instances, an alignment pattern, to create a branded andexclusive optical barcode that is machine-readable using softwareprovided by the entity.

In some embodiments, once a wearable device is paired with a hostdevice, images captured by the wearable device are locked to the pairedaccount and may only be transferred to devices associated with theaccount or uploaded to a network using an account associated with thehost device at the time of the pairing. If a wearable device is pairedwith a new host associated with a new account, new content becomesassociated with the new account. In some embodiments, old content isautomatically secured or deleted. If the content is secured, it may bereleased by pairing with a host device associated with the old account.In some embodiments, user selectable options may enable such accountlocking or may enable removal of such account locking such that anycontent may be shared with any account or host device.

Further, in some embodiments, a wearable device maintains reserveresources to perform the analysis of a custom pattern system in order toenable pairing even when the memory of the device has reached a criticallevel. For example, a pair of glasses with a camera system may includememory to store images and video captured by the camera system. Due tothe size and structure of the wearable device, the available memory maybe limited. When a threshold amount of memory is used, the devicereturns an error indicating that the memory is full and unavailable tocapture additional content (e.g., images or videos.) In such a state,the device maintains a memory reserve to capture and process details ofimages to determine if an optical code indicating a device pairing is inuse. This enables the device to pair with a host device for wirelesstransfer of content data to the host device and to free up the memory.

FIG. 1A is a network diagram depicting a network system 100 having aclient-server architecture configured for exchanging data over anetwork, according to one embodiment. For example, the network system100 may be a messaging system where clients communicate and exchangedata within the network system 100. The data may pertain to variousfunctions (e.g., sending and receiving text and media communication,determining geolocation, etc.) and aspects associated with the networksystem 100 and its users. Although illustrated herein as client-serverarchitecture, other embodiments may include other network architectures,such as peer-to-peer or distributed network environments.

As shown in FIG. 1A, the network system 100 includes a social messagingsystem 130. The social messaging system 130 is generally based on athree-tiered architecture, consisting of an interface layer 124, anapplication logic layer 126, and a data layer 128. As is understood byskilled artisans in the relevant computer and Internet-related arts,each module or engine shown in FIG. 1A represents a set of executablesoftware instructions and the corresponding hardware (e.g., memory andprocessor) for executing the instructions. To avoid obscuring theinventive subject matter with unnecessary detail, various functionalmodules and engines that are not germane to conveying an understandingof the inventive subject matter have been omitted from FIG. 1A. In someembodiments, additional functional modules and engines may be used witha social messaging system, such as that illustrated in FIG. 1A, tofacilitate additional functionality that is not specifically describedherein. Furthermore, the various functional modules and engines depictedin FIG. 1A may reside on a single server computer or may be distributedacross several server computers in various arrangements. Moreover,although the social messaging system 130 is depicted in FIG. 1A as athree-tiered architecture, the inventive subject matter is by no meanslimited to such an architecture.

As shown in FIG. 1A, the interface layer 124 consists of interfacemodules (e.g., a web server) 140, which receive requests from variousclient-computing devices and servers, such as client devices 110executing client applications 112, and third party servers 120 executingthird party applications 122. In response to received requests, theinterface modules 140 communicate appropriate responses to requestingdevices via a network 104. For example, the interface modules 140 canreceive requests such as Hypertext Transfer Protocol (HTTP) requests orother web-based Application Programming Interface (API) requests.

The client devices 110 can execute conventional web browser applicationsor applications (also referred to as “apps”) that have been developedfor a specific platform to include any of a wide variety of mobilecomputing devices and mobile-specific operating systems (e.g., IOS™,ANDROID™, WINDOWS® PHONE). In an example, the client devices 110 areexecuting the client applications 112. The client applications 112 canprovide functionality to present information to a user 106 andcommunicate via the network 104 to exchange information with the socialmessaging system 130. Each of the client devices 110 can comprise acomputing device that includes at least a display and communicationcapabilities with the network 104 to access the social messaging system130. The client devices 110 comprise, but are not limited to, remotedevices, work stations, computers, general purpose computers, Internetappliances, hand-held devices, wireless devices, portable devices,wearable computers, cellular or mobile phones, personal digitalassistants (PDAs), smart phones, tablets, ultrabooks, netbooks, laptops,desktops, multi-processor systems, microprocessor-based or programmableconsumer electronics, game consoles, set-top boxes, network PCs,minicomputers, and the like. Users 106 can include a person, a machine,or other means of interacting with the client devices 110. In someembodiments, the users 106 interact with the social messaging system 130via the client devices 110.

As shown in FIG. 1A, the data layer 128 has one or more database servers132 that facilitate access to information storage repositories ordatabases 134. The databases 134 are storage devices that store datasuch as member profile data, social graph data (e.g., relationshipsbetween members of the social messaging system 130), and other userdata.

An individual can register with the social messaging system 130 tobecome a member of the social messaging system 130. Once registered, amember can form social network relationships (e.g., friends, followers,or contacts) on the social messaging system 130 and interact with abroad range of applications provided by the social messaging system 130.

The application logic layer 126 includes various application logicmodules 150, which, in conjunction with the interface modules 140,generate various user interfaces with data retrieved from various datasources or data services in the data layer 128. Individual applicationlogic modules 150 may be used to implement the functionality associatedwith various applications, services, and features of the socialmessaging system 130. For instance, a social messaging application canbe implemented with one or more of the application logic modules 150.The social messaging application provides a messaging mechanism forusers of the client devices 110 to send and receive messages thatinclude text and media content such as pictures and video. The clientdevices 110 may access and view the messages from the social messagingapplication for a specified period of time (e.g., limited or unlimited).In an example, a particular message is an ephemeral message accessibleto a message recipient for a predefined duration (e.g., specified by amessage sender) that begins when the particular message is firstaccessed. After the predefined duration elapses, the message is deletedand is no longer accessible to the message recipient. The system maymanage deletion of such an ephemeral message based on an ephemeralmessage deletion trigger that is generated by the device capturing andsending the image, or by the system.

As illustrated in FIG. 1A, the social messaging system 130 or the clientapplications 112 include a custom pattern system 160 that providesfunctionality to identify and decode optical barcodes that employ customfunctional patterns. In various embodiments, the custom pattern system160 can be implemented as a standalone system and is not necessarilyincluded in the social messaging system 130. In some embodiments, theclient devices 110 include a portion of the custom pattern system 160(e.g., a portion of the custom pattern system 160 may be includedindependently or in the client applications 112). In embodiments wherethe client devices 110 include a portion of the custom pattern system160, the client devices 110 can work alone or in conjunction with theportion of the custom pattern system 160 included in a particularapplication server or included in the social messaging system 130.

FIG. 1B illustrates an alternative network system 101 that may be usedwith certain embodiments. Network system 101 includes social messagingsystem 130 with interface modules 140, application logic modules 150,database servers 132, and databases 134, as well as client devices 110operating client applications 112, just as in network system 100.Network system 101, however, additionally includes client companiondevices 114 connected to client devices 110. Client companion devices114 may be wearable devices such as glasses, visors, watches, or othernetwork enabled items. Client companion devices may also be any devicedescribed herein that accesses a network such as network 104 via anotherdevice such as client device 110. Client companion devices 114 includeimage sensors 116, wireless input and output (I/O) 117, and elements ofcustom pattern system 160. Client companion devices 114 may include oneor more processors, a display, a battery, and a memory, but may havelimited processing and memory resources. In such embodiments, clientdevice 110 and/or server computing devices used for social messagingsystem 130 may be used via network connections to provide remoteprocessing and memory resources for client companion devices 114. In oneembodiment, for example, client companion device 114 may be a pair ofnetwork enabled glasses, such as the glasses of FIG. 13 . In order toprovide security for wireless I/O 117 in communication with clientdevice 110, custom pattern system 160 may interact with image sensors116 to identify optical patterns as part of secure pairing operations asdetailed below.

FIG. 2 is a block diagram 200 of the custom pattern system 160. Thecustom pattern system 160 is shown to include a communication module210, a presentation module 220, a finder module 230, an alignment module240, a decoder module 250, an action module 260, and an encoder module270. All, or some, of the modules 210-270 communicate with each other,for example, via a network coupling, shared memory, and the like. Eachmodule of the modules 210-270 can be implemented as a single module,combined into other modules, or further subdivided into multiplemodules. Other modules not pertinent to example embodiments can also beincluded, but are not shown.

The communication module 210 provides various communicationsfunctionality. For example, the communication module 210 receives,accesses, or otherwise obtains image data of an image from a userdevice. In a specific example, the communication module 210 receivessubstantially real-time image data from a camera sensor of a smart phone(e.g., a single frame of image data or a continuous stream of framescaptured by a camera sensor of the smart phone). The communicationmodule 210 exchanges network communications with the database servers132, the client devices 110, and the third party servers 120. Theinformation retrieved by the communication module 210 includes dataassociated with the user (e.g., member profile data from an onlineaccount or social network service data) or other data to facilitate thefunctionality described herein.

The presentation module 220 provides various presentation and userinterface functionality operable to interactively present and receiveinformation to and from the user. For instance, the presentation module220 is utilizable to present user interfaces generated in response todecoding the optical barcode. In other instances, the presentationmodule 220 generates user interfaces that include optical barcode(s). Invarious embodiments, the presentation module 220 presents or causespresentation of information (e.g., visually displaying information on ascreen, acoustic output, haptic feedback). The process of interactivelypresenting information is intended to include the exchange ofinformation between a particular device and the user. The user mayprovide input to interact with the user interface in many possiblemanners, such as alphanumeric, point based (e.g., cursor), tactile, orother input (e.g., touch screen, tactile sensor, light sensor, infraredsensor, biometric sensor, microphone, gyroscope, accelerometer, or othersensors). The presentation module 220 provides many other userinterfaces to facilitate functionality described herein. The term“presenting” as used herein is intended to include communicatinginformation or instructions to a particular device that is operable toperform presentation based on the communicated information orinstructions.

The finder module 230 provides image processing functionality toidentify, recognize, or detect the custom graphic being employed as afinder pattern in the optical barcode. For example, the finder module230 extracts and analyzes candidate shape features or candidate contourcharacteristics from image data of the image received from the userdevice (e.g., the client devices 110). The finder module 230 determinessatisfaction of various rules or criteria associated with the extractedcandidate shape features. The finder module 230 compares the extractedcandidate shape features with reference shape features of the customgraphic, or another reference image, to identify the custom graphicincluded in the image. The finder module 230 can employ a wide varietyof schemes and techniques to extract the candidate shape features fromthe image data of the image and subsequently identify the custom graphicbased on an analysis of the candidate shape features.

The alignment module 240 provides image processing functionality todetermine an alignment of the optical barcode using the custom graphic.The custom pattern system 160 can use the alignment to facilitatedecoding of data encoded in the optical barcode. In this way, the customgraphic functions as an alignment pattern for the optical barcode. Forexample, the alignment module 240 extracts spatial attributes of thecustom graphic in the image from the image data. In various embodiments,the spatial attributes include at least one of position, orientation,scale, or another spatial aspect of the optical barcode. The alignmentmodule 240 determines an alignment of the optical barcode based on thespatial attributes (e.g., a particular orientation of the opticalbarcode). In an example, the alignment module 240 can determine analignment including position and orientation based on the spatialattributes and generate a transformed image according to the alignment.The custom pattern system 160 can then use the transformed image todecode data encoded in a portion of the transformed image.

The decoder module 250 provides functionality to decode data encoded inthe image using the spatial attributes or the determined alignment ofthe custom graphic in the image. For instance, the decoder module 250can decode the data encoded in the image from an image transformedaccording to the spatial attributes of the custom graphic extracted fromimage data. In an embodiment, the decoder module 250 detects markings(e.g., high contrast dots, squares, or other marks in the image)representing data encoded in a portion of the image from the image data.In a specific example, the decoder module 250 employs a Reed-Solomonerror correction scheme to decode data encoded in the image. TheReed-Solomon error correction scheme allows for a successful or validdecoding even when a certain percentage of data could not be decodedfrom the optical barcode (e.g., damaged bits or incorrectly decodedbits). In some embodiments, the user or an administrator of the custompattern system 160 configures a tolerance value for an amount of damagedor incorrectly decoded data acceptable when decoding the opticalbarcode. In some embodiments, the decoder module 250 also provides imageprocessing functionality to improve decoding of the optical barcode. Forinstance, the decoder module 250, as well as the alignment module 240,can perform image transforms of the image (e.g., perform imagesharpening, de-noise processing, other digital filtering, or other imageprocessing techniques to improve decoding accuracy).

The action module 260 provides functionality to perform a variety ofactions based on decoding the data encoded in the image. For example,the data encoded in a portion of the image can indicate a particularaction or include information to be used in conjunction with aparticular action. In a specific example, the data encoded in a portionof the image can comprise a user name, or other user identification, ofa member of a social networking service and based on decoding the username, the action module 260 can perform an action on the socialnetworking service corresponding to the user name (e.g., sending amessage to the member associated with the user name). In someembodiments, the action module 260 performs an action specific to aparticular app that scans the image (e.g., a function available to auser of the app but otherwise unavailable). In some instances, theaction module 260 performs the action without communicating with anexternal server (e.g., an action locally performed on the user devicethat scanned the optical code).

The encoder module 270 provides functionality to generate and encodedata into an optical barcode that employs the custom graphic as one ormore functional patterns (e.g., generating optical codes). As discussedabove in connection with the decoder module 250, in a specific example,the encoder module 270 can employ a technique such as Reed-Solomon errorcorrection to encode data. In an example embodiment, the encoder module270 renders a machine-readable arrangement of marks that represents thedata to be encoded. The encoder module 270 can then generate themachine-readable optical barcode using the rendered arrangement of marksand the custom graphic to be used as a functional pattern.

In an example embodiment, a custom pattern system receives image datarepresenting an image from a user device. For example, the custompattern system receives the image data from an optical sensor (e.g., acamera sensor) of a smart phone of the user. In various embodiments, theimage data from the user device is received in response to auser-initiated image capture, a periodic monitoring of image data beingdetected by the optical sensor of the user device, an access of storedimage data, or a combination thereof. A portion of the image data caninclude data representing an optical barcode employing a custom graphicfor a particular functional pattern (e.g., a finder pattern). In somescenarios, the image data includes extraneous or irrelevant data alongwith the data pertaining to the optical barcode (e.g., an image of anoptical barcode includes a background that is not pertinent to decodingthe optical barcode). In a specific example, the optical sensor of theuser device captures an image of a promotional poster that includes aparticular optical barcode. The image of the promotional poster caninclude the particular optical barcode along with irrelevant portions ofthe promotional poster or background that surrounds the particularoptical barcode.

After the custom pattern system receives the image data, the custompattern system searches the image data of the image for the customgraphic to determine whether the image includes the optical barcode.That is to say, the custom graphic is used as a finder pattern forrecognition, identification, or detection of the optical barcode withinthe image. In an example embodiment, the custom pattern system searchesfor the custom graphic by extracting a candidate shape feature, ormultiple candidate shape features, from the image data. For example, thecustom pattern system performs an edge detection technique, or anotherimage processing technique, to identify the candidate shape feature,such as a contour line of the image. The custom pattern system thendetermines whether the candidate shape feature satisfies shape featurerules or criteria. For instance, if a particular candidate shape featureis a contour line, the custom pattern system can determine whether thecontour line is an enclosed line that encircles a portion of the image.Consistent with some embodiments, the shape feature rules filter outirrelevant or extraneous candidate shape features or candidate shapefeatures with a low probability of being the custom graphic.

In response to the candidate shape feature satisfying the shape featurerules, the custom pattern system identifies the custom graphic bycomparing the candidate shape feature with a reference shape feature ofthe custom graphic. For example, the custom pattern system can comparean area or size of the candidate shape feature with a reference area orsize of the reference shape feature. In this example, the custom patternsystem identifies the custom graphic based on a match or near match(e.g., a percentage match above a threshold) between the candidate shapefeature and the reference shape feature. In this way, the custom patternsystem uses the custom graphic as a finder pattern to identify thepresence of the optical barcode within a portion of the image.

In further example embodiments, the custom graphic functions as analignment pattern to facilitate the custom pattern system decoding thedata encoded in the optical barcode. In an example embodiment, thecustom pattern system extracts spatial attributes of the custom graphicin the image from the image data. For example, the custom pattern systemextracts a position, scale, or orientation of the custom graphic fromthe image data. The custom pattern system decodes data encoded in theimage from the image data using the spatial attributes of the customgraphic in the image. For instance, the custom pattern system canperform an image transform using the spatial attributes (e.g., ade-skew, a rotation, a scale, or another type of image transform) toimprove detectability/readability of data encoded in a portion of theimage. In this way, the custom pattern system uses the custom graphic asan alignment pattern to facilitate decoding the optical barcode.

Accordingly, the custom pattern system uses the custom graphic as afunctional pattern of the optical barcode without utilizing conventionalfunctional patterns. Using the custom graphic as a functional patternallows for an aesthetically pleasing design and can provide exclusivityto a particular software application as the functional pattern does notnecessarily conform to an open standard and thus is readable exclusivelyby the particular software application.

FIGS. 3A and 3B are diagrams illustrating examples of optical barcodesemploying the custom graphic for a finder pattern or an alignmentpattern (e.g., optical codes). Diagram 300 shows an example opticalbarcode that includes a custom graphic 310 (e.g., a company logo), andmarkings 320 that represent data encoded into the optical barcode. Inthis example, the custom graphic 310 is a company logo. Custom graphicsused as a functional pattern can include designs with multiple paths,multiple polygons, multiple aesthetic elements, or other designfeatures.

As shown in the diagram 300, the markings 320 are dots that are arrangedin a pattern with a particular spacing or positioning readable by amachine. Although the diagram 300 shows the markings 320 as dots, othershapes and marks can be employed (e.g., squares or asymmetric shapes ofvarious geometries). The markings 320 can be arranged in a uniformpattern or a non-uniform pattern. In some instances, the marks can be ofdifferent sizes or a uniform size. Additionally, the markings 320 can bein a predetermined arrangement or an arrangement that is dynamicallydeterminable when decoding data from the markings. In some embodiments,the custom graphic 310 and the markings 320 can be surrounded by abounding shape, such as an outer box 325. Although the outer box 325 ofthe diagram 300 is shown as a square with rounded corners, the outer box325 can be in the form of a variety of other shapes with variousgeometries. Diagram 330 in FIG. 3B shows another example optical barcodethat employs the custom graphic for a finder pattern or an alignmentpattern. The diagram 330 shows the optical barcode with markingsexcluded from within the custom graphic. In these and other embodiments,the space internal to the custom graphic may be reserved for other uses.For example, a picture, graphic, animation, annotation, or imageselected by a user may be inserted.

Turning now to FIG. 4A, a diagram illustrating an example of identifyingand decoding the optical barcode employing the custom graphic for afinder pattern or an alignment pattern is shown. FIG. 4A is an overviewof a particular example embodiment of identifying and decoding theoptical barcode using the custom graphic. FIG. 4B illustrates a wearabledevice that may observe an optical code displayed on a host device inorder to initiate pairing between the wearable device and the hostdevice. Additional details and alternative implementations are discussedin connection with the figures to follow. In FIGS. 4A and 4B, a hostdevice 414 displays an optical barcode. A user 410 wearing a wearabledevice 431 may use sensors of wearable device 431 to capture an imageincluding optical barcode, as presented on a display of device 414.Callout 412 portrays an enlarged view of a portion of the scene capturedby the sensors of wearable device 431 that includes the optical barcodefrom the display of device 414 in at least a portion of the scene.

In an example embodiment, the user device 414 captures an image of thescene that includes the optical barcode as displayed on the host device.The custom pattern system 160 receives the image data representing theimage from the user device 414. In this example embodiment, the custompattern system 160 is included in the wearable device 431 and in thedevice 414, where the device 414 uses the custom pattern system 160 togenerate an optical barcode and display it, and device 431 uses custompattern system 160 to identify the barcode generated by device 414(e.g., applications executing on glasses and a smart phone of the user410). Callout 416 portrays example image processing performed by thefinder module 230 to identify the custom graphic in the image and usethe custom graphic as an alignment pattern for decoding data included inthe optical barcode. In the callout 416, the finder module 230 extractscandidate shape features from the image data of the image. Subsequently,the finder module 230 determines if the candidate features meet certainrules and criteria to filter out irrelevant shape features or shapefeatures that have a low probability of being the custom graphic. Thefinder module 230 can then compare the candidate shape features thatmeet the shape feature criteria or rules with reference shape featuresof the custom graphic. In an example, the finder module 230 identifiesthe custom graphic based on a match between the candidate shape featuresand the reference shape feature (e.g., a match score that exceeds athreshold).

Subsequent to the finder module 230 identifying the custom graphic, thecustom pattern system 160 can use the custom graphic as an alignmentpattern for decoding. For instance, the alignment module 240 extractsspatial attributes of the custom graphic in the image and compares theextracted spatial attributes to reference spatial attributes todetermine an alignment of the custom graphic. The alignment module 240or the decoder module 250 may then generate a transformed image of theimage according to the alignment (e.g., a rotation or de-skew) as shownin callout 418. After generating the transformed image, the decodermodule 250 decodes the data encoded in a portion of the transformedimage as shown in callout 420. In the callout 420, the dots of theoptical barcode are transformed into data shown as ones for dots andzeros for non-dots, although this is merely an illustrative example andother schemes can be employed. In this way, the custom pattern system160 uses the custom graphic included in the optical barcode as one ormore functional patterns such as a finder pattern or an alignmentpattern.

FIG. 5 then describes details of one embodiment for device pairing, inaccordance with embodiments described herein. FIG. 5 illustratesoperations and communications performed by a phone 502 (e.g., a hostdevice), a wearable device 504, and a phone 506 (e.g., a miscellaneouslocal or secondary host device). In the embodiment of FIG. 5 , in anapplication operating on a host device phone 502, a user navigates to aprofile page or otherwise interacts with phone 502 such that phone 502displays an optical code in operation 510. The device 504 then recordsvideo or captures an image using the wearable device 504, where thecaptured content includes an image of the optical code being displayedby phone 502. The wearable device detects the optical code in operation512, and begins advertising (e.g., broadcasting wireless data) inoperations 514A/B. In some embodiments, this broadcasting is performedusing Bluetooth Low Energy™.

Any host devices (e.g., smartphones) nearby, such as phone 506, that areexecuting the application and are in a state to detect a genericadvertisement message (e.g., at a profile screen) will detect theadvertisement communication, and then display a special pairing opticalcode. In some embodiments, this may be a different size or shape thatthe initial optical code (e.g., 9×9 vs. 18×18.) In some embodiments, thepairing optical code includes 20 random bits used as a verificationcode, 10 randomly generated bits for advertisement code, and 10 errorcorrection bits. In FIG. 5 , phone 502 and 506 both receive the genericadvertising message (e.g., a message not customized for a specific hostdevice or phone) and respond with operations 516 and 518 where eachphone displays a pairing code (e.g., a second optical code).

Each phone 502 and 506 then begins scanning for additional advertisementcommunications that match the advertisement code from the pairingoptical code while displaying the pairing codes of operations 516 and518.

The wearable device then captures and decodes the pairing optical codefrom operation 516 using the image sensor, in operation 520. This may besimilar to the capture and decoding of the profile optical code. Thewearable device ignores the paring code from operation 518 from phone506. The wearable device advertising communication is then updated toinclude the advertisement code from the pairing optical code displayedby the host device in operation 522. The host device phone 502 that hasbeen scanning for the matching advertisement identifies that thewearable device is broadcasting the matching advertisement code, andthen allows the connection between the host device and the wearabledevice in operation 524. Phone 506 may receive the advertisement code ofoperation 522, but will not match the details from pairing codeoperation 518, and so will ignore the code of operation 522, and willcontinue scanning until the user of 506 navigates away from the profilepage or otherwise shifts to a state without scanning for advertisingcodes.

In some embodiments, the wearable device and host device then carry outan application-level elliptic curve Diffie-Hellman (ECDH) exchange inoperation 526 to establish a shared secret (e.g. Diffie-Hellman key(DHKey)). Further communications are then encrypted via counter modeencryption using this shared secret in operations 528. In someembodiments, the host device application computes a hash messageauthentication code (HMAC) based on SHA-256 of DHKey, Rv and a hardcoded128-bit SaltA. The application sends this from the host device phone 502to the wearable device 504 in operation 530, and the wearable device 504verifies that this matches the expected value or is otherwise correct inoperation 532.

In some embodiments, the wearable device further computes the HMAC basedon SHA-256 of DHKey, Rv and a hardcoded 128-bit SaltB. The wearabledevice then sends this to the host device in operation 534, and theapplication on the host device verifies that this matches the expectedvalue or is otherwise correct in operation 536. The handshake is thencompleted in operation 538. Completion of the handshake may additionallybe used to trigger different actions in device 504 and associated hostdevice phone 502. In some embodiments, for example, images captured ondevice 504 are limited to being communicated to phone 502. Attempts toperform a new handshake with a different phone will trigger deletion ofimages stored in a shared memory of device 504 that were captured whiledevice 504 was paired with phone 502. Similarly, in some embodiments,transmission of images or video clips from device 504 may be limited tophone 502, and further encrypted such that an application on device 504and phone 502 use the secure information to limit viewing of content tophone 502 for images captured by device 504 while paired with phone 502.Phone 502 may “unlock” such images and allow them to be communicated asephemeral messages with an associated deletion trigger via a networksuch as social messaging system 130. In some embodiments, the DHKey orother information from the handshaking process is needed to decrypt orotherwise unlock the images for distribution.

In some embodiments, the pairing involves a comparison handshakecompliant with H7.3 of the Bluetooth LE™ core specification.

Once this initial pairing has been performed, both devices may storepairing information to enable the secure connection to be reestablishedwithout initiating another optical code pairing. In some embodiments,such information is stored at the wearable device for only a single hostdevice at a time, such that pairing with a different host devicerequires optical code pairing to reestablish the connection with thefirst host device.

FIG. 6 is a flow diagram illustrating further example operations foridentifying the optical barcode using the custom functional pattern.Such operations may be used by a wearable device such as device 504 whendetecting optical codes in operations 512 and 520. In FIG. 6 , thefinder module 230 determines that the candidate shape feature satisfiesthe shape feature rule.

At operation 610, the finder module 230 determines that the candidateshape feature comprises an enclosed line from the image data. That is tosay, the shape feature rule comprises a path rule and the finder module230 determines that the candidate shape feature satisfies the path rule.The finder module 230 can employ a variety to techniques to determinethat the candidate shape feature satisfies the path rule.

At operation 630, the finder module 230 determines whether the candidateshape feature is an enclosed line by determining that the candidateshape feature encircles a portion of the image by having a path thatstarts at a particular point and returns to the same particular point.In an example embodiment, if the candidate shape feature does notsatisfy the path rule (indicated by “no” in FIG. 6 ), no furtheranalysis of the candidate shape feature is performed and the findermodule 230 analyzes another candidate shape feature or performs nofurther operations. Alternatively, at operation 640, if the findermodule 230 determines that the candidate shape feature satisfies thepath rule (indicated by “yes” in FIG. 6 ), the subsequent operationsproceed with detecting the optical code.

To illustrate the concepts of FIG. 6 , FIG. 7 is a diagram 700illustrating an example of identifying the optical barcode using thecustom functional pattern. In the diagram 700, the image 710 is anexample image that is received or accessed from the user device. Theimage 720 is an example image portraying example candidate shapefeatures 730. For instance, the finder module 230 performs an edgedetection image processing on the image 710 to derive the image 720.From the image 720, the finder module 230 identifies the candidate shapefeatures 730.

Callout 740 shows a particular candidate shape feature of the candidateshape features 730. The callout 740 shows a contour line 750(illustrated as a dotted line) of the particular candidate shapefeature, a path 760, and a point 770 of the particular candidate shapefeature. In the callout 740, the finder module 230 determines that thepath rule is met if the path 760 that starts at the point 770 can followthe contour line 750 and return to the point 770. In the diagram 700,the particular candidate shape feature shown in the callout 740 doessatisfy the path rule since the path 760 can follow the contour line 750and return to the point 770.

FIG. 8 is a flow diagram illustrating further example operations foridentifying the optical barcode using the custom functional pattern. InFIG. 8 , the finder module 230 determines that the candidate shapefeature satisfies the shape feature rule. At operation 810, the findermodule 230 calculates an area value or size approximation of thecandidate shape feature. For example, the finder module 230 uses a proxyshape such as a polygon (e.g., a square, a rectangle, or aquadrilateral) or a non-polygonal shape (e.g., an ellipse) toapproximate the shape of the candidate shape feature. The finder module230 fits or nearly fits the proxy shape to the outer edges or outerperimeter of the candidate shape feature so that the proxy shape isrepresentative of an area of the candidate shape feature. Subsequently,the finder module 230 calculates the area value of the proxy shape todetermine the area value or size approximation of the candidate shapefeature. In some embodiments, the finder module 230 employs such atechnique (e.g., polygonal area approximation) to avoid acomputationally expensive area calculation of the candidate shapefeature in situations where the candidate shape feature is likely to becomplex in shape (e.g., an area calculation for a non-uniform orirregular shaped feature is typically more computationally expensive).In some embodiments, other techniques such as pixel based counting canbe employed to determine the area value.

At operation 820, the finder module 230 determines an area score or sizescore of the candidate shape feature. The finder module 230 determinesthe area score by comparing the area value of the candidate shapefeature with a reference area value. In some embodiments, the referencearea value comprises an area value of a corresponding proxy shape fittedto a reference image of the custom graphic (e.g., the area value of aproxy shape fitted to the ghost logo from a front view perspective). Inother embodiments, the reference area value comprises the area value ofthe custom graphic (e.g., the area value of the ghost logo). The findermodule 230 calculates the area score, for example, by determining amatch percentage between the candidate shape feature area value and thereference area value. The finder module 230 can employ a wide variety ofother schemes and techniques to calculate the area score.

At operation 830, the finder module 230 determines whether the areascore exceeds a threshold. The threshold can be predefined ordynamically determined (e.g., statistically determined based on arolling historical average of scans).

At operation 840, based on the area score exceeding the threshold(indicated by “yes” in FIG. 8 ), the finder module 230 determines thatthe candidate shape feature satisfies the area rule and proceeds tosubsequent operations. In another example embodiment, the finder module230 compares the area score to an area range to satisfy the area rule(e.g., greater than a particular value and less than a particularvalue). If the area score does not exceed the threshold (indicated by“no” in FIG. 8 ) in operation 830, the finder module 230 analyzesanother candidate shape feature or no further operations are performed,according to an example embodiment. In some example embodiments, thefinder module 230 uses the determination of whether the candidate shapefeature satisfies the shape feature rules as a filter (e.g., to removeor skip candidate shape features that are unlikely to be the customgraphic) to identify candidate shape features to be further analyzed inthe process of identifying the custom graphic in the image.

To further illustrate the concepts of FIG. 8 , FIG. 9 is a diagram 900illustrating an example of identifying the optical barcode using thecustom functional pattern. In the diagram 900, image 902 is an exampleimage that is received from the user device. Callout 904 shows thespatial orientation of the image 902. In this example, the image 902 isportrayed and being seen from a front right perspective. The image 902includes optical barcode 906. In this example, the optical barcode 906employs the custom graphic as a functional pattern.

Callout 908 shows an enlarged portion of the image 902 that includes thecandidate shape feature being analyzed by the finder module 230 toidentify the custom graphic. In the callout 908, the polygon 910 (e.g.,a quadrilateral) is shown fitted to a perimeter of the candidate shapefeature. Area value 912 is the area of the polygon 910.

Callout 914 shows a reference image of the custom graphic. Callout 916shows the spatial orientation of the reference image. In this example,the reference image is shown from the front view perspective. Polygon918 is shown fitted to a perimeter of the reference image. Referencearea value 920 is the area of the polygon 918. Although FIG. 9 showspolygons 910 and 918 as quadrilaterals, the finder module 230 can employother shapes such as a square or a shape that follows or traces acontour of the candidate shape feature (e.g., an n-sided polygon orsmooth fitted shape that follows contour points of the candidate shapefeature).

The finder module 230 compares the area value 912 with the referencearea value 920 to determine that the candidate shape feature satisfiesthe area rule. Another candidate shape feature of the image 902, such asone of the musical notes of the image 902, would not have an area valuethat is similar to the reference area value and therefore would notsatisfy the area rule. In this way, the finder module 230 can quicklyremove or skip certain candidate shape features that are unlikely to beidentified as the custom graphic.

FIG. 10 then illustrates a method 1000 for device pairing with opticalcodes. In one embodiment, method 1000 is performed by a wearable devicesuch as glasses 31 of FIG. 13 . In other embodiments, other suchwearable devices may be used. In some embodiments, method 1000 isperformed by elements of a wearable device, including processors, imagesensors, and communication circuitry of a wearable device. In someembodiments, method 1000 is instantiated as instructions stored in anon-transitory storage medium that may be executed by one or moreprocessors to cause a device to perform the operations of method 1000.

Method 100 begins with operation 1005 where a wearable device capturesan image including a first optical code from the first host device. Insome embodiments, this capturing may involve multiple frames of a videoclip. In other embodiments, this capturing may involve a single imagecaptured as a picture. In some embodiments, this data is stored in ashared memory of the wearable device, along with other pieces of contentcaptured and stored for future communication as ephemeral messages via acommunication network. In other embodiments, the shared memory may befull, and the image is stored in a volatile or reserve memory whilebeing processed. The image is then deleted after the data is extractedto free the reserve memory for additional images that may be processedwhile the shared memory is full.

The first optical code is then decoded, using one or more processorscoupled to the image sensor of wearable device, in operation 1010. Thisdecoding may involve various transformations or calculations torecognize an optical code, including identifying custom patterns for anoptical code as described herein. Then in operation 1015, in response tothe first optical code, the wearable device begins broadcasting apairing advertisement using a wireless transceiver coupled to the one ormore processors. As described herein, this broadcast may use BluetoothLE™ or may use any other wireless communication standards, such as anInstitute of Electrical and Electronic Engineering (IEEE) 802.11standard communication.

Following this, in operation 1020, the wireless device proceeds withcapturing, in response to the pairing advertisement and using the imagesensor, a second optical code from the first host device, with thesecond optical code comprising a host pairing advertisement code. Inoperation 1025, the wireless device gets the advertisement code bydecoding the second optical code to determine the host pairingadvertisement code. Then, in response to the second optical code, inoperation 1030, the wireless device begins broadcasting a second pairingadvertisement including the host pairing advertisement code.

In various embodiments, the host pairing advertisement code is used toestablish communications between the wearable device and the host devicein a variety of different ways. In some embodiments, this information isused in establishing secure communications, and then generating andstoring a set of handshaking information that allows a secure connectionto be reestablished without the use of optical codes, in operation 1035.As described above, in some embodiments, this involves the use ofDHKeys. In other embodiments, other secure communications may be used.For example, in some embodiments, the DHKeys may be stored as part ofthe set of handshaking information. In some embodiments, wirelesscommunication identifiers and one or more secure access codes to enablereestablishment of the secure connection are stored as the handshakinginformation, such that inputs on the user interface of the wearabledevice or the host device may be used to initiate reestablishing of thesecure connection between the host device and the wearable device. Insome embodiments, only a single set of handshaking information may beassociated with a wearable device at any given time, such that thewearable device will only respond to the most recent set of handshakinginformation, and any previous handshaking information associated with awearable device will be ignored (e.g. deleted from memory.) In otherembodiments, multiple sets of handshaking information may be associatedwith either a host device or a wearable device, and upon a userinterface input to reestablish a connection, the user may select aparticular pairing, or the device may cycle through all sets of storedhandshaking information in an attempt to reestablish a connection withany nearby device that has previously been paired.

In some embodiments, the pairing advertisement comprises a genericadvertising signal broadcast such that a plurality of local host devicesreceive and decode the pairing advertisement, with each local hostdevice of the plurality of local host devices displaying a resultinghost pairing advertisement code. As described above, this allows anylocal host that is listening to detect the initial pairingadvertisement.

In some embodiments, additional operations are performed including, inresponse to the second pairing advertisement, establishing acommunication channel between the first host device and the wearabledevice; communicating, via the communication channel, a first key fromthe wearable device to the first host device; receiving, via thecommunication channel, a second key at the wearable device from the hostdevice; and establishing an encrypted connection using the first key andthe second key.

This may operate in some embodiments such that the encrypted connectionis established using an application-level elliptic curve ECDH exchangewith the first key and the second key to establish a shared secret, andwhere the communication channel is established using a Bluetooth LowEnergy connection between the first host device and the wearable device.

Further such embodiments may operate by receiving, at the wearabledevice via the encrypted connection, a first HMAC based on the sharedsecret, wherein the shared secret comprises a DHKey; verifying, usingthe one or more processors of the wearable device, the first HMAC;generating, using the one or more processors of the wearable device inresponse to verification of the first HMAC, a second HMAC based on theDHKey; and transmitting the second HMAC from the wearable device to thefirst host device.

As part of decoding the first optical code, various embodiments mayperform operations such as receiving image data of an image from a userdevice; extracting a candidate shape feature of the image from the imagedata; determining that the candidate shape feature satisfies a shapefeature criterion; in response to the candidate shape feature satisfyingthe shape feature criterion, identifying a custom symbol in the image bycomparing the candidate shape feature with a reference shape feature ofthe custom symbol; in response to identifying the custom symbol,extracting, using a hardware processor of a machine, a geometryattribute of the custom symbol in the image from the image data; anddecoding data encoded in a portion of the image from the image datausing the geometry attribute of the custom symbol in the image.

Some such operations for processing an optical code may additionallyinclude: calculating a size approximation of the candidate shape featurefrom the image data; determining a size score for the candidate shapefeature by comparing the size approximation with a reference size of thecustom symbol; determining that the candidate shape feature satisfies asize criterion based on the size score exceeding a threshold, whereinthe shape feature criterion comprises the size criterion; determiningfrom the image data that the candidate shape feature comprises anenclosed line, wherein the enclosed line encircles a portion of theimage by having a path that starts at a particular point and returns tothe particular point; or determining that the candidate shape featuresatisfies a path criterion based on the candidate shape featurecomprising the enclosed line, wherein the shape feature criterioncomprises the path criterion.

Similarly, some embodiments may detect optical codes where a geometryattribute comprises at least one of a position, scale, or orientation ofthe custom symbol in the image. Additional optical code detectionoperations may include extracting a distinctive feature of the customsymbol from the image data, the distinctive feature being indicative ofan identification of the custom symbol; determining an orientation ofthe custom symbol in the image by comparing the distinctive feature witha reference distinctive feature of the custom symbol; and decoding thedata encoded in the image using the orientation and a position of thecustom symbol in the image.

In some embodiments, images captured on the wearable device areencrypted using the secure data or handshaking information. This mayfunction to “lock” the images and video content to the paired hostdevice, such that the paired device needs to view the content capturedby the wearable device. In some embodiments, a wearable device may storecontent associated with a first host while paired to a second host. Insome such embodiments, the handshaking information prevents the datafrom being transferred to the second host device. In other embodiments,the data may be transferred, but is not viewable at the second hostdevice. In still further embodiments, the content associated with thefirst host device is automatically deleted when a pairing to a secondhost device occurs.

FIG. 11 then illustrates a method 1100 for device pairing with opticalcodes. In one embodiment, method 1100 is performed by a host device suchas phone 502 of FIG. 5 or device 1200 of FIGS. 12A-B. In otherembodiments, other such host devices may be used. In some embodiments,method 1100 is instantiated as instructions stored in a non-transitorystorage medium, that may be executed by one or more processors to causea device to perform the operations of method 1100. In some embodiments,method 1100 comprises operations performed by a host device inconjunction with corresponding operations of method 1000 being performedby a wearable device, with the host and wearable devices communicatingwith each other as part of the operations. Method 1100 begins withoperation 1105 where the host device generates and displays an imageincluding a first optical code. In some embodiments, this may be inresponse to a user interface election, in some embodiments, an initialscreen of an application operating on a host device involve presentationof an optical code on a screen of the host device. In still furtherembodiments, various sensors may initiate display of an optical code inan image. For example, a clock or alarm may periodically initiate such adisplay, or sensors detecting proximity to or a signal from a wearabledevice may initiate an initial optical code.

During display of the optical code in operation 1105, the host devicelistens for a pairing advertisement, and in operation 1110, the hostdevice receives and decodes a pairing advertisement from a wearabledevice. This pairing advertisement may be generic, without any detailassociated with the host device.

In operation 1115, the host device generates and displays a secondoptical code from the first host device, with the second optical codecomprising a host pairing advertisement code. This host pairingadvertisement code includes information specific to the host generatingand displaying the code (e.g., in contrast to the generic pairingadvertisement.)

Again, while displaying the second optical code, the host device listensfor a pairing advertisement. In operation 1120, the host device receivesa second pairing advertisement. The second pairing advertisement is notgeneric and includes the host pairing advertisement code or a relatedcode derived from the host pairing advertisement code. In operation1120, the host device decodes a second pairing advertisement includingthe host pairing advertisement code and uses this information to verifythat the correct wearable device is identified for pairing.

In operation 1125, in response to decoding the second pairingadvertisement, the host device carries out an application-level ECDHexchange to establish a shared secret and establish a secure connectionbetween the wearable device and the host device. In operation 1130, thisconnection is used to generate and store a set of handshakinginformation to enable a secure connection to be reestablished withoutoptical code pairing.

As described above, this information may be used to lock contentcaptured by a wearable device to the host device that is paired with thewearable device when the content is captured. The host device may use anapplication to “unlock” such images and allow them to be communicated asephemeral messages with an associated deletion trigger via a networksuch as social messaging system 130. In some embodiments, the DHKey orother information from the handshaking process is needed to decrypt orotherwise unlock the images for distribution.

Additionally, in some embodiments, the host device executes the aboveoperations as part of a shared application environment with the wearabledevice. In such embodiments, the wearable device and the host deviceeach execute separate portions of the shared application environment,with the host device executing a host portion and the wearable deviceexecuting the wearable device portion. The host portion may be anapplication executed in response to a user interface input received froma host device user. The wearable device portion may be an integral partof the wearable device that always operates. When the host applicationdisplays the optical code, the wearable device application executesoperations to recognize optical codes in a pattern that is expectedwithin the context of the shared application environment. This mayinclude the use of custom images or code patterns, and associated rulesfor recognizing the code patterns at the wearable device in a way thatis configured to limited processor usage at the wearable device, giventhe lower battery and processing resource of the wearable devicecompared with the host device.

Additionally, as described above, the host application may operate toinitiate a particular wireless listening protocol while optical codesare displayed. In such embodiments, the application configures theprocessors to initiate a wireless listening protocol only while theoptical code is displayed, or for a limited time period associated withdisplay of the optical code. Each new presentation of an optical codemay be associated with a corresponding limited wireless listeningperiod. If the host device does not receive the appropriate pairingadvertisement during the listening period, the pairing processterminates. Thus, while a device displaying a first optical code mayrespond to an initial generic pairing advertisement, display a secondoptical code in response to the initial generic pairing advertisement,and receive a second pairing advertisement, no connection will be madeif the second pairing advertisement does not include the correct code,and the host device will stop listening after the associated listeningperiod. In some embodiments, the listening period is limited to a setamount of time (e.g. 10 seconds, 15 seconds, etc.) after the display ofan optical code is initiated to conserve power and prevent unwantedcommunications. In other embodiments, the listening period matches thedisplay of the optical code (e.g. will end when a sleep trigger or othertrigger causes the optical code to no longer be displayed.) In otherembodiments, any such time period for wireless listening associated withdisplay of an optical code may be used. Similarly, for a single host,different listening periods may be used for the initial optical codewhile listening for a generic pairing advertisement, and the secondoptical code while listening for a pairing advertisement with a hostpairing code.

While particular operations are described above, it will be apparentthat operations may be repeated, or intervening operations may bepresent, such that variations on the above methods are possible withinthe scope of the described embodiments for device pairing with opticalcodes.

FIG. 12A illustrates an example mobile device 1200 that may be used as ahost device for pairing with a wearable device using optical codes asdescribed herein. Inputs and adjustments to any system operationdescribed herein may be performed using touch screen inputs 1292 withindevice display area 1290 by a user 1294. Additionally, selection of userinterface inputs to display an initial optical code and a subsequentpairing code may use touch screen input 1292. Additional inputs mayinitiate transmission of advertising communication and initiate scanningfor communications from a device to be paired.

FIG. 12B illustrates an example mobile device 1200 executing a mobileoperating system (e.g., IOS™, ANDROID™, WINDOWS® Phone, or other mobileoperating systems), consistent with some embodiments. In one embodiment,the mobile device 1200 includes a touch screen operable to receivetactile data from a user 1202. For instance, the user 1202 mayphysically touch 1204 the mobile device 1200, and in response to thetouch 1204, the mobile device 1200 may determine tactile data such astouch location, touch force, or gesture motion. In various exampleembodiments, the mobile device 1200 displays a home screen 1206 (e.g.,Springboard on IOS™) operable to launch applications or otherwise managevarious aspects of the mobile device 1200. In some example embodiments,the home screen 1206 provides status information such as battery life,connectivity, or other hardware statuses. The user 1202 can activateuser interface elements by touching an area occupied by a respectiveuser interface element. In this manner, the user 1202 interacts with theapplications of the mobile device 1200. For example, touching the areaoccupied by a particular icon included in the home screen 1206 causeslaunching of an application corresponding to the particular icon.

Many varieties of applications (also referred to as “apps”) can beexecuted on the mobile device 1200, such as native applications (e.g.,applications programmed in Objective-C, Swift, or another suitablelanguage running on IOS™, or applications programmed in Java running onANDROID™), mobile web applications (e.g., applications written inHypertext Markup Language-5 (HTML5)), or hybrid applications (e.g., anative shell application that launches an HTML5 session). For example,the mobile device 1200 includes a messaging app, an audio recording app,a camera app, a book reader app, a media app, a fitness app, a filemanagement app, a location app, a browser app, a settings app, acontacts app, a telephone call app, or other apps (e.g., gaming apps,social networking apps, biometric monitoring apps). In another example,the mobile device 1200 includes a social messaging app 1208 such asSNAPCHAT® that, consistent with some embodiments, allows users toexchange ephemeral messages that include media content. In this example,the social messaging app 1208 can incorporate aspects of embodimentsdescribed herein.

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules can constitute eithersoftware modules (e.g., code embodied on a machine-readable medium) orhardware modules. A “hardware module” is a tangible unit capable ofperforming certain operations and can be configured or arranged in acertain physical manner. In various example embodiments, one or morecomputer systems (e.g., a standalone computer system, a client computersystem, or a server computer system) or one or more hardware modules ofa computer system (e.g., a processor or a group of processors) can beconfigured by software (e.g., an application or application portion) asa hardware module that operates to perform certain operations asdescribed herein.

In some embodiments, a hardware module can be implemented mechanically,electronically, or any suitable combination thereof. For example, ahardware module can include dedicated circuitry or logic that ispermanently configured to perform certain operations. For example, ahardware module can be a special-purpose processor, such as aField-Programmable Gate Array (FPGA) or an Application SpecificIntegrated Circuit (ASIC). A hardware module may also includeprogrammable logic or circuitry that is temporarily configured bysoftware to perform certain operations. For example, a hardware modulecan include software executed by a general-purpose processor or otherprogrammable processor. Once configured by such software, hardwaremodules become specific machines (or specific components of a machine)uniquely tailored to perform the configured functions and are no longergeneral-purpose processors. It will be appreciated that the decision toimplement a hardware module mechanically, in dedicated and permanentlyconfigured circuitry, or in temporarily configured circuitry (e.g.,configured by software) can be driven by cost and time considerations.

Accordingly, the phrase “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. As used herein,“hardware-implemented module” refers to a hardware module. Consideringembodiments in which hardware modules are temporarily configured (e.g.,programmed), each of the hardware modules need not be configured orinstantiated at any one instance in time. For example, where a hardwaremodule comprises a general-purpose processor configured by software tobecome a special-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware modules) at different times. Softwareaccordingly configures a particular processor or processors, forexample, to constitute a particular hardware module at one instance oftime and to constitute a different hardware module at a differentinstance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules can be regarded as being communicatively coupled. Where multiplehardware modules exist contemporaneously, communications can be achievedthrough signal transmission (e.g., over appropriate circuits and buses)between or among two or more of the hardware modules. In embodiments inwhich multiple hardware modules are configured or instantiated atdifferent times, communications between such hardware modules may beachieved, for example, through the storage and retrieval of informationin memory structures to which the multiple hardware modules have access.For example, one hardware module can perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module can then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules can also initiate communications with input oroutput devices and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein can beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors constitute processor-implemented modulesthat operate to perform one or more operations or functions describedherein. As used herein, “processor-implemented module” refers to ahardware module implemented using one or more processors.

Similarly, the methods described herein can be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method can be performed by one or more processors orprocessor-implemented modules. Moreover, the one or more processors mayalso operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an Application ProgramInterface (API)).

The performance of certain of the operations may be distributed amongthe processors, not only residing within a single machine, but deployedacross a number of machines. In some example embodiments, the processorsor processor-implemented modules can be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented modules are distributed across a number ofgeographic locations.

The modules, methods, applications and so forth described in conjunctionwith the figures above are implemented in some embodiments in thecontext of a machine and an associated software architecture. Thesections below describe representative software architecture(s) andmachine (e.g., hardware) architecture that are suitable for use with thedisclosed embodiments.

Software architectures are used in conjunction with hardwarearchitectures to create devices and machines tailored to particularpurposes. For example, a particular hardware architecture coupled with aparticular software architecture will create a mobile device, such as amobile phone, tablet device, or so forth. A slightly different hardwareand software architecture may yield a smart device for use in the“internet of things,” while yet another combination produces a servercomputer for use within a cloud computing architecture. Not allcombinations of such software and hardware architectures are presentedhere as those of skill in the art can readily understand how toimplement the various embodiments in different contexts from thedisclosure contained herein.

FIG. 13 shows aspects of certain embodiments illustrated by a frontperspective view of glasses 31. Glasses 31 may, in some embodiments, bea client companion device 114 or wearable device that is used to captureimages including optical codes and pairing codes as part of operationsto create secure communications between the glasses 31 and a hostdevice. The glasses 31 can include a frame 32 made from any suitablematerial such as plastic or metal, including any suitable shape memoryalloy. The frame 32 can have a front piece 33 that can include a firstor left lens, display, or optical element holder 36; and a second orright lens, display, or optical element holder 37 connected by a bridge38. The front piece 33 additionally includes a left end portion 41 and aright end portion 42. A first or left optical element 44 and a second orright optical element 43 can be provided within respective left andright optical element holders 36, 37. Each of the optical elements 43,44 can be a lens, a display, a display assembly, or a combination of theforegoing. Any of the display assemblies disclosed herein can beprovided in the glasses 31.

Frame 32 additionally includes a left arm or temple piece 46 and asecond arm or temple piece 47 coupled to the respective left and rightend portions 41, 42 of the front piece 33 by any suitable means such asa hinge (not shown), so as to be coupled to the front piece 33, orrigidly or fixably secured to the front piece so as to be integral withthe front piece 33. Each of the temple pieces 46 and 47 can include afirst portion 51 that is coupled to the respective end portion 41 or 42of the front piece 33 and any suitable second portion 52, such as acurved or arcuate piece, for coupling to the ear of the user. In oneembodiment, the front piece 33 can be formed from a single piece ofmaterial, so as to have a unitary or integral construction.

Glasses 31 can include a computing device, such as computer 61, whichcan be of any suitable type so as to be carried by the frame 32 and, inone embodiment, of a suitable size and shape so as to be at leastpartially disposed in one of the temple pieces 46 and 47. In oneembodiment, as illustrated in FIG. 13 , the computer 61 is sized andshaped similar to the size and shape of one of the temple pieces 46 and47 and is thus disposed almost entirely if not entirely within thestructure and confines of such temple pieces 46 and 47. In oneembodiment, the computer 61 can be disposed in both of the temple pieces46, 47. The computer 61 can include one or more processors with memory,wireless communication circuitry, and a power source. In certainembodiments, the computer 61 comprises low-power circuitry, high-speedcircuitry, and a display processor. Various other embodiments mayinclude these elements in different configurations or integratedtogether in different ways.

The computer 61 additionally includes a battery 62 or other suitableportable power supply. In one embodiment, the battery 62 is disposed inone of the temple pieces 46 or 47. In the glasses 31 shown in FIG. 13 ,the battery 62 is shown as being disposed in left temple piece 46 andelectrically coupled using connection 74 to the remainder of thecomputer 61 disposed in the right temple piece 47. The one or more inputand output devices can include a connector or port (not shown) suitablefor charging a battery 62 accessible from the outside of frame 32, awireless receiver, transmitter or transceiver (not shown), or acombination of such devices.

Glasses 31 include cameras 69. Although two cameras are depicted, otherembodiments contemplate the use of a single or additional (i.e., morethan two) cameras. In various embodiments, glasses 31 may include anynumber of input sensors or peripheral devices in addition to cameras 69.Front piece 33 is provided with an outward facing, forward-facing, orfront or outer surface 66 that faces forward or away from the user whenthe glasses 31 are mounted on the face of the user, and an oppositeinward-facing, rearward-facing, or rear or inner surface 67 that facesthe face of the user when the glasses 31 are mounted on the face of theuser. Such sensors can include inwardly-facing video sensors or digitalimaging modules such as cameras that can be mounted on or providedwithin the inner surface 67 of the front piece 33 or elsewhere on theframe 32 so as to be facing the user, and outwardly-facing video sensorsor digital imaging modules such as cameras 69 that can be mounted on orprovided with the outer surface 66 of the front piece 33 or elsewhere onthe frame 32 so as to be facing away from the user. Such sensors,peripheral devices, or peripherals can additionally include biometricsensors, location sensors, or any other such sensors. In additionalembodiments, similar elements may be presented as visors, within helmetor google based systems, in vehicle HUD displays, or in any other suchdevice.

FIG. 14 is a block diagram 2000 illustrating a representative softwarearchitecture 2002, which may be used in conjunction with varioushardware architectures herein described. FIG. 14 is merely anon-limiting example of a software architecture and it will beappreciated that many other architectures may be implemented tofacilitate the functionality described herein. The software architecture2002 may be executing on hardware such as machine 2100 of FIG. 15 thatincludes, among other things, processors 2110, memory/storage 2130, andinput/output (I/O) components 2150. A representative hardware layer 2004is illustrated and can represent, for example, the machine 2100 of FIG.15 . The representative hardware layer 2004 comprises one or moreprocessing units 2006 having associated executable instructions 2008.Executable instructions 2008 represent the executable instructions ofthe software architecture 2002, including implementation of the methods,modules and so forth in the figures and description above. Hardwarelayer 2004 also includes memory and storage modules 2010, which alsohave executable instructions 2008. Hardware layer 2004 may also compriseother hardware as indicated by 2012 which represents any other hardwareof the hardware layer 2004, such as the other hardware illustrated aspart of machine 2100.

In the example architecture of FIG. 14 , the software 2002 may beconceptualized as a stack of layers where each layer provides particularfunctionality. For example, the software 2002 may include layers such asan operating system 2014, libraries 2016, frameworks/middleware 2018,applications 2020, and presentation layer 2022. Operationally, theapplications 2020 or other components within the layers may invoke APIcalls 2024 through the software stack and receive a response, returnedvalues, and so forth illustrated as messages 2026 in response to the APIcalls 2024. The layers illustrated are representative in nature and notall software architectures have all layers. For example, some mobile orspecial purpose operating systems may not provide theframeworks/middleware layer 2018, while others may provide such a layer.Other software architectures may include additional or different layers.

The operating system 2014 may manage hardware resources and providecommon services. The operating system 2014 may include, for example, akernel 2028, services 2030, and drivers 2032. The kernel 2028 may act asan abstraction layer between the hardware and the other software layers.For example, the kernel 2028 may be responsible for memory management,processor management (e.g., scheduling), component management,networking, security settings, and so on. The services 2030 may provideother common services for the other software layers. The drivers 2032may be responsible for controlling or interfacing with the underlyinghardware. For instance, the drivers 2032 may include display drivers,camera drivers, BLUETOOTH® drivers, flash memory drivers, serialcommunication drivers (e.g., Universal Serial Bus (USB) drivers), WI-FI®drivers, audio drivers, power management drivers, and so forth dependingon the hardware configuration. In an example embodiment, the operatingsystem 2014 includes imaging service 2033 that can provide imageprocessing services, such as hardware accelerated image processing, orimage capture services, such as low level access to optical sensors oroptical sensor data.

The libraries 2016 may provide a common infrastructure that may beutilized by the applications 2020 or other components or layers. Thelibraries 2016 typically provide functionality that allows othersoftware modules to perform tasks in an easier fashion than to interfacedirectly with the underlying operating system 2014 functionality (e.g.,kernel 2028, services 2030 or drivers 2032). The libraries 2016 mayinclude system 2034 libraries (e.g., C standard library) that mayprovide functions such as memory allocation functions, stringmanipulation functions, mathematic functions, and the like. In addition,the libraries 2016 may include API libraries 2036 such as medialibraries (e.g., libraries to support presentation and manipulation ofvarious media format such as MPREG4, H.264, MP3, AAC, AMR, JPG, or PNG),graphics libraries (e.g., an OpenGL framework that may be used to render2D and 3D in a graphic content on a display), database libraries (e.g.,SQLite that may provide various relational database functions), weblibraries (e.g., WebKit that may provide web browsing functionality),and the like. The libraries 2016 may also include a wide variety ofother libraries 2038 to provide many other APIs to the applications 2020and other software components/modules. In an example embodiment, thelibraries 2016 include imaging libraries 2039 that provide imageprocessing or image capture functionality that can be utilized by thecustom pattern system 160.

The frameworks 2018 (also sometimes referred to as middleware) mayprovide a higher-level common infrastructure that may be utilized by theapplications 2020 or other software components/modules. For example, theframeworks 2018 may provide various graphic user interface (GUI)functions, high-level resource management, high-level location services,and so forth. The frameworks 2018 may provide a broad spectrum of otherAPIs that may be utilized by the applications 2020 or other softwarecomponents/modules, some of which may be specific to a particularoperating system or platform. In an example embodiment, the frameworks2018 include an image processing framework 2022 and an image captureframework 2023. The image processing framework 2022 can providehigh-level support for image processing functions that can be used inaspects of the custom pattern system 160. Similarly, the image captureframework 2023 can provide high-level support for capture images andinterfacing with optical sensors.

The applications 2020 include built-in applications 2040 or third partyapplications 2042. Examples of representative built-in applications 2040may include, but are not limited to, a contacts application, a browserapplication, a book reader application, a location application, a mediaapplication, a messaging application, or a game application. Third partyapplications 2042 may include any of the built-in applications as wellas a broad assortment of other applications. In a specific example, thethird party application 2042 (e.g., an application developed using theANDROID™ or IOS™ software development kit (SDK) by an entity other thanthe vendor of the particular platform) may be mobile software running ona mobile operating system such as IOS™, ANDROID™, WINDOWS® Phone, orother mobile operating systems. In this example, the third partyapplication 2042 may invoke the API calls 2024 provided by the mobileoperating system such as operating system 2014 to facilitatefunctionality described herein. In an example embodiment, theapplications 2020 include a messaging application 2043 that includes thecustom pattern system 160 as part of the application. In anotherembodiment, the applications 2020 include a stand-alone application 2045that includes the custom pattern system 160.

The applications 2020 may utilize built-in operating system functions(e.g., kernel 2028, services 2030 or drivers 2032), libraries (e.g.,system 2034, APIs 2036, and other libraries 2038), frameworks/middleware2018 to create user interfaces to interact with users of the system.Alternatively, or additionally, in some systems interactions with a usermay occur through a presentation layer, such as presentation layer 2044.In these systems, the application/module “logic” can be separated fromthe aspects of the application/module that interact with a user.

Some software architectures utilize virtual machines. In the example ofFIG. 14 , this is illustrated by virtual machine 2048. A virtual machinecreates a software environment where applications/modules can execute asif they were executing on a hardware machine (such as the machine ofFIG. 15 , for example). A virtual machine is hosted by a host operatingsystem (operating system 2014 in FIG. 15 ) and typically, although notalways, has a virtual machine monitor 2046, which manages the operationof the virtual machine as well as the interface with the host operatingsystem (i.e., operating system 2014). A software architecture executeswithin the virtual machine such as an operating system 2050, libraries2052, frameworks/middleware 2054, applications 2056 or presentationlayer 2058. These layers of software architecture executing within thevirtual machine 2048 can be the same as corresponding layers previouslydescribed or may be different.

FIG. 15 is a block diagram illustrating components of a machine 2100,according to some example embodiments, able to read instructions from amachine-readable medium (e.g., a machine-readable storage medium) andperform any one or more of the methodologies discussed herein.Specifically, FIG. 15 shows a diagrammatic representation of the machine2100 in the example form of a computer system, within which instructions2116 (e.g., software, a program, an application, an applet, an app, orother executable code) for causing the machine 2100 to perform any oneor more of the methodologies discussed herein can be executed.Additionally, or alternatively, the instruction can implement thecommunication module 210, the presentation module 220, the finder module230, the alignment module 240, the decoder module 250, the action module260, or the encoder module 270 of FIG. 2 ., and so forth. Theinstructions transform the general, non-programmed machine into aparticular machine programmed to carry out the described and illustratedfunctions in the manner described. In alternative embodiments, themachine 2100 operates as a standalone device or can be coupled (e.g.,networked) to other machines. In a networked deployment, the machine2100 may operate in the capacity of a server machine or a client machinein a server-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine 2100 cancomprise, but not be limited to, a server computer, a client computer, apersonal computer (PC), a tablet computer, a laptop computer, a netbook,a set-top box (STB), a PDA, an entertainment media system, a cellulartelephone, a smart phone, a mobile device, a wearable device (e.g., asmart watch), a smart home device (e.g., a smart appliance), other smartdevices, a web appliance, a network router, a network switch, a networkbridge, or any machine capable of executing the instructions 2116,sequentially or otherwise, that specify actions to be taken by themachine 2100. Further, while only a single machine 2100 is illustrated,the term “machine” shall also be taken to include a collection ofmachines 2100 that individually or jointly execute the instructions 2116to perform any one or more of the methodologies discussed herein.

The machine 2100 can include processors 2110, memory/storage 2130, andI/O components 2150, which can be configured to communicate with eachother such as via a bus 2102. In an example embodiment, the processors2110 (e.g., a Central Processing Unit (CPU), a Reduced Instruction SetComputing (RISC) processor, a Complex Instruction Set Computing (CISC)processor, a Graphics Processing Unit (GPU), a Digital Signal Processor(DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), anotherprocessor, or any suitable combination thereof) can include, forexample, processor 2112 and processor 2114 that may execute instructions2116. The term “processor” is intended to include multi-core processorthat may comprise two or more independent processors (sometimes referredto as “cores”) that can execute instructions contemporaneously. AlthoughFIG. 15 shows multiple processors, the machine 2100 may include a singleprocessor with a single core, a single processor with multiple cores(e.g., a multi-core processor), multiple processors with a single core,multiple processors with multiples cores, or any combination thereof.

The memory/storage 2130 can include a memory 2132, such as a mainmemory, or other memory storage, and a storage unit 2136, bothaccessible to the processors 2110 such as via the bus 2102. The storageunit 2136 and memory 2132 store the instructions 2116 embodying any oneor more of the methodologies or functions described herein. Theinstructions 2116 can also reside, completely or partially, within thememory 2132, within the storage unit 2136, within at least one of theprocessors 2110 (e.g., within the processor’s cache memory), or anysuitable combination thereof, during execution thereof by the machine2100. Accordingly, the memory 2132, the storage unit 2136, and thememory of the processors 2110 are examples of machine-readable media.

As used herein, the term “machine-readable medium” means a device ableto store instructions and data temporarily or permanently and mayinclude, but is not be limited to, random-access memory (RAM), read-onlymemory (ROM), buffer memory, flash memory, optical media, magneticmedia, cache memory, other types of storage (e.g., Erasable ProgrammableRead-Only Memory (EEPROM)) or any suitable combination thereof. The term“machine-readable medium” should be taken to include a single medium ormultiple media (e.g., a centralized or distributed database, orassociated caches and servers) able to store instructions 2116. The term“machine-readable medium” shall also be taken to include any medium, orcombination of multiple media, that is capable of storing instructions(e.g., instructions 2116) for execution by a machine (e.g., machine2100), such that the instructions, when executed by one or moreprocessors of the machine 2100 (e.g., processors 2110), cause themachine 2100 to perform any one or more of the methodologies describedherein. Accordingly, a “machine-readable medium” refers to a singlestorage apparatus or device, as well as “cloud-based” storage systems orstorage networks that include multiple storage apparatus or devices. Theterm “machine-readable medium” excludes signals per se.

The I/O components 2150 can include a wide variety of components toreceive input, provide output, produce output, transmit information,exchange information, capture measurements, and so on. The specific I/Ocomponents 2150 that are included in a particular machine will depend onthe type of machine. For example, portable machines such as mobilephones will likely include a touch input device or other such inputmechanisms, while a headless server machine will likely not include sucha touch input device. It will be appreciated that the I/O components2150 can include many other components that are not shown in FIG. 15 .The I/O components 2150 are grouped according to functionality merelyfor simplifying the following discussion and the grouping is in no waylimiting. In various example embodiments, the I/O components 2150 caninclude output components 2152 and input components 2154. The outputcomponents 2152 can include visual components (e.g., a display such as aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), a projector, or a cathode ray tube (CRT)),acoustic components (e.g., speakers), haptic components (e.g., avibratory motor, resistance mechanisms), other signal generators, and soforth. The input components 2154 can include alphanumeric inputcomponents (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, or other pointinginstruments), tactile input components (e.g., a physical button, a touchscreen that provides location and force of touches or touch gestures, orother tactile input components), audio input components (e.g., amicrophone), and the like.

In further example embodiments, the I/O components 2150 can includebiometric components 2156, motion components 2158, environmentalcomponents 2160, or position components 2162 among a wide array of othercomponents. For example, the biometric components 2156 can includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram basedidentification), and the like. The motion components 2158 can includeacceleration sensor components (e.g., an accelerometer), gravitationsensor components, rotation sensor components (e.g., a gyroscope), andso forth. The environmental components 2160 can include, for example,illumination sensor components (e.g., a photometer), temperature sensorcomponents (e.g., one or more thermometers that detect ambienttemperature), humidity sensor components, pressure sensor components(e.g., a barometer), acoustic sensor components (e.g., one or moremicrophones that detect background noise), proximity sensor components(e.g., infrared sensors that detect nearby objects), gas sensorcomponents (e.g., machine olfaction detection sensors, gas detectionsensors to detect concentrations of hazardous gases for safety or tomeasure pollutants in the atmosphere), or other components that mayprovide indications, measurements, or signals corresponding to asurrounding physical environment. The position components 2162 caninclude location sensor components (e.g., a Global Positioning System(GPS) receiver component), altitude sensor components (e.g., altimetersor barometers that detect air pressure from which altitude may bederived), orientation sensor components (e.g., magnetometers), and thelike.

Communication can be implemented using a wide variety of technologies.The I/O components 2150 may include communication components 2164operable to couple the machine 2100 to a network 2180 or devices 2170via a coupling 2182 and a coupling 2172, respectively. For example, thecommunication components 2164 include a network interface component orother suitable device to interface with the network 2180. In furtherexamples, communication components 2164 include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, BLUETOOTH®components (e.g., BLUETOOTH® Low Energy), WI-FI® components, and othercommunication components to provide communication via other modalities.The devices 2170 may be another machine or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a USB).

Moreover, the communication components 2164 can detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 2164 can include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as a Universal Product Code (UPC) barcode, multi-dimensional bar codes such as a Quick Response (QR) code,Aztec Code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code,Uniform Commercial Code Reduced Space Symbology (UCC RSS)-2D bar codes,and other optical codes), acoustic detection components (e.g.,microphones to identify tagged audio signals), or any suitablecombination thereof. In addition, a variety of information can bederived via the communication components 2164, such as location viaInternet Protocol (IP) geo-location, location via WI-FI® signaltriangulation, location via detecting a BLUETOOTH® or NFC beacon signalthat may indicate a particular location, and so forth.

In various example embodiments, one or more portions of the network 2180can be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a WI-FI®network, another type of network, or a combination of two or more suchnetworks. For example, the network 2180 or a portion of the network 2180may include a wireless or cellular network, and the coupling 2182 may bea Code Division Multiple Access (CDMA) connection, a Global System forMobile communications (GSM) connection, or other type of cellular orwireless coupling. In this example, the coupling 2182 can implement anyof a variety of types of data transfer technology, such as SingleCarrier Radio Transmission Technology (1xRTT), Evolution-Data Optimized(EVDO) technology, General Packet Radio Service (GPRS) technology,Enhanced Data rates for GSM Evolution (EDGE) technology, thirdGeneration Partnership Project (3GPP) including 3G, fourth generationwireless (4G) networks, Universal Mobile Telecommunications System(UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability forMicrowave Access (WiMAX), Long Term Evolution (LTE) standard, othersdefined by various standard setting organizations, other long rangeprotocols, or other data transfer technology.

The instructions 2116 can be transmitted or received over the network2180 using a transmission medium via a network interface device (e.g., anetwork interface component included in the communication components2164) and utilizing any one of a number of well-known transfer protocols(e.g., HTTP). Similarly, the instructions 2116 can be transmitted orreceived using a transmission medium via the coupling 2172 (e.g., apeer-to-peer coupling) to devices 2170. The term “transmission medium”shall be taken to include any intangible medium that is capable ofstoring, encoding, or carrying the instructions 2116 for execution bythe machine 2100, and includes digital or analog communications signalsor other intangible medium to facilitate communication of such software.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader scope of embodiments of the present disclosure. Such embodimentsof the inventive subject matter may be referred to herein, individuallyor collectively, by the term “invention” merely for convenience andwithout intending to voluntarily limit the scope of this application toany single disclosure or inventive concept if more than one is, in fact,disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, modules, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be implementedas a combined structure or resource. Similarly, structures andfunctionality presented as a single resource may be implemented asseparate resources. These and other variations, modifications,additions, and improvements fall within a scope of embodiments of thepresent disclosure as represented by the appended claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

What is claimed is:
 1. An apparatus of a host device comprising: atleast one processor; and a memory storing instructions that, whenexecuted by the processor, configure the apparatus of the host device toperform operations comprising: causing an indication of an optical codeto be displayed on a display of the host device, the optical codeindicating a host pairing advertisement code; and receiving a pairingadvertisement from a mobile device, the pairing advertisement comprisingan indication of the host pairing advertisement code.
 2. The apparatusof claim 1 wherein the receiving further comprises: receiving, via afirst wireless communication, a pairing advertisement from a mobiledevice, the pairing advertisement comprising an indication of the hostpairing advertisement code.
 3. The apparatus of claim 1, wherein theoptical code is a first optical code, and wherein before the causing theoperations further comprise: causing an indication of a second opticalcode to be displayed on the display of the host device.
 4. The apparatusof claim 3, wherein the causing the indication of the second opticalcode further comprises: in response to receiving a user input toinitiate pairing with the mobile device, causing, the indication of thesecond optical code to be displayed on the display of the host device,the second optical code soliciting the generic pairing advertisementfrom the mobile device.
 5. The apparatus of claim 3, wherein the pairingadvertisement is a first pairing advertisement, and wherein theoperations further comprise: receiving a second pairing advertisementfrom the mobile device.
 6. The apparatus of claim 1, wherein theoperations further comprise: in response to a determination that thepairing advertisement comprises the indication of the host pairingadvertisement code, sending to the mobile device, a first key.
 7. Theapparatus of claim 6, wherein the operations further comprise: receivinga second key from the wearable device, the second key based on the firstkey.
 8. The apparatus of claim 7, wherein the operations furthercomprise: establishing an encrypted connection using the first key andthe second key.
 9. The apparatus of claim 8 wherein the operationsfurther comprise: transmitting a wireless communication to the wearabledevice soliciting transmission of content from the mobile device to thehost device via the encrypted connection.
 10. The apparatus of claim 8,wherein the encrypted connection is established using an elliptic curveDiffie-Hellman (ECDH) exchange with the first key and the second key toestablish a shared secret.
 11. The apparatus of claim 8, wherein theoperations further comprise: generating a first keyed-hash messageauthentication code (HMAC) based on the shared secret, wherein theshared secret comprises a Diffie-Hellman Key (DHKey); transmitting thefirst HMAC to the wearable device; and processing a second HMAC based onthe DHKey received from the wearable device.
 12. The apparatus of claim1, wherein the operations further comprise: an antenna coupled to thememory, and wherein the receiving further comprises: receiving, via theantenna, the pairing advertisement from the mobile device, the pairingadvertisement comprising the indication of the host pairingadvertisement code.
 13. The apparatus of claim 1, wherein the opticalcode comprises a custom reference shape associated with a shape featurerule.
 14. The apparatus of claim 1, wherein the mobile device is a firstmobile device and the pairing advertisement is a first pairingadvertisement, and wherein the operations further comprise: receiving asecond pairing advertisement from a second wearable device; and inresponse to a determination that the second pairing advertisement doesnot indicate the host pairing advertisement code, refraining frominitiating a secure communication with the second mobile device.
 15. Theapparatus of claim 1 further comprising: an image sensor coupled to thememory and configured to capture images, and wherein the receiving thepairing advertisement from the mobile device, further comprises:capturing an image comprising the pairing advertisement from a displayof the mobile device; and determining the pairing advertisement from theimage.
 16. A non-transitory computer-readable storage medium includinginstructions that, when processed by an apparatus of a host device,configure the apparatus to perform operations comprising: causing anindication of an optical code to be displayed on a display of the hostdevice, the optical code indicating a host pairing advertisement code;and receiving a pairing advertisement from a mobile device, the pairingadvertisement comprising an indication of the host pairing advertisementcode.
 17. The non-transitory computer-readable storage medium of claim16 wherein the receiving further comprises: receiving, via a firstwireless communication, a pairing advertisement from a mobile device,the pairing advertisement comprising an indication of the host pairingadvertisement code.
 18. The non-transitory computer-readable storagemedium of claim 16, wherein the optical code is a first optical code,and wherein before the causing, the operations further comprise: causingan indication of a second optical code to be displayed on the display ofthe host device.
 19. A method performed on an apparatus of a hostdevice, the method comprising: causing an indication of an optical codeto be displayed on a display of the host device, the optical codeindicating a host pairing advertisement code; and receiving a pairingadvertisement from a mobile device, the pairing advertisement comprisingan indication of the host pairing advertisement code.
 20. The method ofclaim 19 wherein the receiving further comprises: receiving, via a firstwireless communication, a pairing advertisement from a mobile device,the pairing advertisement comprising an indication of the host pairingadvertisement code.